Augmented Reality Projects in Automotive and Aerospace Industry

In 2003 the International Symposium on Mixed and Augmented Reality (ISMAR) was accompanied by a workshop on Potential Industrial Applications (PIA). The organizers wisely called it "potential" because the real use of augmented reality (AR) in an industrial context is still in its infancy. Our own experience in this field clearly supports this viewpoint. We have been actively involved in the research, development and deployment of AR systems in the automotive, aviation and astronautics industries for more than five years and have developed and implemented AR systems in a wide variety of environments. In this paper we have selected ten AR projects from those we have managed and implemented in the past to examine the main challenges faced and to share some of the lessons learned. We will conclude with some guidelines for successfully deploying AR in an industrial context. Introduction: Augmented Reality in an industrial context Bringing research results out of the laboratory and into an industrial context is always a challenge. And if this process eventually leads to success on the market it is usually called innovation. Innovations in the technological area of augmented reality are rare. On the one hand research and development (R&D) is still in its early days. On the other hand the academic and industry partners both agree that there is huge potential for the technology in a broad variety of applications. As a result various attempts to bring R&D and “real world use” of AR together have been made and are still top of the list for potential innovations. It can be said that the application of augmented reality in an industrial context started with Boeing’s wire bundle assembly project in the early 90’s (Mizell, 2001) followed by several smaller projects until the end of the last century. While numerous academic projects evolved in the following years, industrial augmented reality (IAR) IEEE COMPUTER GRAPHICS AND APPLICATIONS FINAL MANUSCRIPT November / December 2005 Page 2 of 14 Regenbrecht_etal_AR_in_AutomotiveAerospace.doc applications are still rare. In some cases, AR technology was applied successfully in certain use cases. For instance in supporting welding processes (Echtler et al., 2003) or in some training scenarios (see Doerner et al., 2002). To date there have been two major initiatives for AR innovation. The Mixed Reality Systems Laboratory in Japan, with its focus set on the development of mixed reality prototype applications comprising hardware and software, has demonstrated the potential for the real-world use of AR (see Tamura, Yamamoto, & Katayama, 2001). The successes of this project lead to the release of the mixed reality platform, a comprehensive toolkit consisting of display, tracking, and AR software technology. The other initiative being the German project “ARVIKA” lead by Siemens which included the majority of the manufacturing industry in the country as well as selected partners in academia and small and medium enterprises (see Friedrich, 2004). The focus here was on the application of AR in the fields of design, production, and servicing. All these initiatives brought forward various prototypes and demonstrated applications and have therefore been valuable in progressing the field of AR. The lessons learned in these projects have had a strong influence on the direction of AR R&D worldwide. As part of this international community we have developed prototypes of AR applications in the realm of automotive and aerospace industry. A majority of our projects are presented here. Servicing and Maintenance Today’s products are getting more and more complex. The days when a plan of the electrical circuits of a car fit onto one large sheet of paper have long gone, modern high-tech cars now require a database system and state of the art computer equipment for electric and electronic diagnosis. A printout of such a database is as thick as an encyclopedia. How can one bring the right information to the right place at the right time? The use of augmented reality technology seems to be obvious. The service personnel is equipped with a (wearable) computer unit and gets the appropriate information displayed next to or overlaid onto the object being inspected. Not only can this do away with the need for a paper schematic, but a far richer information resource can be provided via online access to dedicated information and multi-media content. The promise is to increase effectiveness (fewer errors) and efficiency (shorter time to complete the task) through the use of context-sensitive, up-to-date, and media-rich information. All major manufacturing enterprises are thinking about how to make use of AR technology in their maintenance and servicing areas. The more complex the product is, the greater the potential benefit of AR. We have selected three areas where we applied AR technology for service personnel. IEEE COMPUTER GRAPHICS AND APPLICATIONS FINAL MANUSCRIPT November / December 2005 Page 3 of 14 Regenbrecht_etal_AR_in_AutomotiveAerospace.doc Figure 1: Servicing projects from left to right: Space station filter change and engine maintenance Space station filter change The European Columbus module of the International Space Station (ISS) is intended to be inhabited (part-time) by astronauts from different countries. As one can imagine this module is a very complex system and requires many maintenance tasks to be undertaken by the astronauts. The augmentation of service information could help decrease the workload. The client for this application (German and European Aerospace industry: DASA RI, EADS Astrium) decided on a step-by-step approach for testing the use of AR technology in space. A fairly simple application scenario was chosen to test the validity of the concept: providing instructions and support for monitoring the state of the air filter and changing the filter if required. Our research and development involved the implementation of an optical see-through solution, the connection of this to a content delivery system, and the identification of opportunities and limitations of AR use in this context. The content delivery was entirely based on an existing virtual reality system. All information displayed was in the form of three-dimensional geometry modeled as a whole or in part beforehand. Together with developers in the client group, a whole wearable AR system was developed consisting of a rugged backpack computer, a modified COTS headmounted display with optical see-through capabilities (Sony Glasstron), and an ultrasonic/inertial tracking system. Although it was intended to display threedimensional content, in the end 2D content aligned to 3D space was provided (see figure 1 above). The system was successfully demonstrated at an international aerospace fair, but the system never made it into space. This was due to the difficulty in meeting the rigorous requirements of aerospace standards, which include being able to withstand extreme operating conditions (e.g. high g-forces), the required unobtrusiveness of the technology within the module (almost no instrumentation of the environment is possible), or the failure-free linkage to the onboard information infrastructure.